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1. (Previously Presented) An image pickup system for 
processing an image signal at each pixel which comprises a 
plurality of color signals, and one or more of the color signals 
are dropped out according to a location of the pixel, the image 
5 pickup system comprising: 

first interpolation means for interpolating the color 
signals dropped-out from the image signals by a first 
interpolation method; 

precision verification means for verifying the interpolation 
10 precision based on the image signals and the color signals 
interpolated by the first interpolation means; and 

second interpolation means for interpolating the color 
signals dropped-out from the image signals by a second 
interpolation method that is different from the first 
15 interpolation method in cases where it is judged that the 

interpolation precision by the first interpolation method is 
insufficient . 



2. (Currently Amended) An image pickup system for 
processing an image signal at each pixel which comprises a 
plurality of color signals, and one or more of the color signals 
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are dropped out according to a location of the pixel, the image 
5 pickup system comprising: 

separation means for separating the image signals into first 
image signals and second image signals based on predetermined 
characteristics relating to the image signals; 

first interpolation means for interpolating the dropped-out 
10 color signals from the first image signals by a first 
interpolation method; 

second interpolation means for interpolating the dropped-out 
color signals from the second image signals by a second 
interpolation method that is different from the first 
15 interpolation means method ; 

precision verification means for verifying the interpolation 
precision based on the first image signals and the color signals 
interpolated by the first interpolation means for ttte regions of 
the first image signals, and verifying the interpolation 
20 precision based on the second image signals and the color signals 
interpolated by the second interpolation means for ttte regions of 
the second image signals; and 

adjustment means for causing interpolation processing of the 
dropped-out color signals to be performed again from the image 
25 signals by the second interpolation means when insufficient 

interpolation was performed by the first interpolation means, and 
for causing interpolation processing of the dropped-out color 
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signals to be performed again from the — image — signals by the first 
interpolation means when insufficient interpolation was performed 
30 by the second interpolation means, in cases where it is judged 
that the interpolation precision is insufficient. 



3. (Previously Presented) An image pickup system for 
processing an image signal at each pixel which comprises a 
plurality of color signals, and one or more of the color signals 
are dropped out according to a location of the pixel, the image 
5 pickup system comprising: 

first interpolation means for interpolating the color 
signals dropped-out from the image signals by a first 
interpolation method; 

second interpolation means for interpolating the color 
10 signals dropped-out from the image signals by a second 
interpolation method that is different from the first 
interpolation method; 

precision verification means for verifying the interpolation 
precision based on the image signals, the color signals 
15 interpolated by the first interpolation means and the color 
signals interpolated by the second interpolation means; and 

selection means for selecting color signals having a higher 
interpolation precision between the color signals interpolated by 
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the first interpolation means and the color signals interpolated 
20 by the second interpolation means. 

4. (Previously Presented) The image pickup system according 
to claim 1, wherein the first interpolation means or second 
interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals, 

edge extraction means for extracting a plurality of edge 
intensities relating to predetermined directions from the pixels 
of interest within the extracted regions, 

weighting calculation means for calculating weighting 
10 coefficients that are normalized from the extracted edge 
intensities, 

interpolation signal calculation means for calculating a 
plurality of interpolation signals relating to predetermined 
directions from the pixels of interest within the extracted 
15 regions, and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest based on a plurality of 
weighting coefficients relating to the predetermined directions 
and a plurality of interpolation signals relating to the 
20 predetermined directions. 
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5. (Previously Presented) The image pickup system according 
to claim 1, wherein the first interpolation means or second 
interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals, and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest within the extracted regions by 
linear interpolation or cubic interpolation. 

6. (Previously Presented) The image pickup system according 
to claim 1, wherein the first interpolation means or second 
interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals, 
correlation calculation means for determining the 
correlation between the respective color signals within the 
extracted regions as a linear equation, and 

calculation means for calculating the dropped-out color 
10 signals based on the image signals and the correlation. 

7. (Previously Presented) The image pickup system according 
to claim 1, wherein the precision verification means comprises: 

correlation calculation means for determining correlation 
information relating to the correlations between the respective 
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color signals for each predetermined region based on the image 

signals and the color signals interpolated by the first 
interpolation means, and 

correlation verification means for verifying the 
interpolation precision based on the correlation information. 

8. (Previously Presented) The image pickup system according 
to claim 1, wherein the precision verification means comprises: 

hue calculation means for determining hue information for 
each pixel based on the image signals and the color signals 
interpolated by the first interpolation means, and 

hue verification means for verifying the interpolation 
precision based on the hue information. 

9. (Previously Presented) The image pickup system according 
to claim 1, wherein the precision verification means comprises: 

edge calculation means for determining edge information for 
each predetermined region based on the image signals and the 
color signals interpolated by the first interpolation means, and 

edge verification means for verifying the interpolation 
precision based on the edge information. 



10. (Previously Presented) The image pickup system 
according to claim 2, wherein the separation means comprises: 
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edge calculation means for determining edge information for 
each predetermined region from the image signals, and 
5 image signal separation means for separating the image 

signals based on the edge information. 

11. (Previously Presented) The image pickup system 
according to claim 2, wherein the separation means comprises: 

correlation calculation means for determining correlation 
information relating to the correlations between the respective 
5 color signals for each predetermined region from the image 
signals, and 

image signal separation means for separating the image 
signals based on the correlation information. 

12. (Previously Presented) The image pickup system 
according to claim 1, further comprising control means for 
controlling the precision verification means and the second 
interpolation means such that the operation of the precision 
verification means and the operation of the second interpolation 
means can be stopped. 

13. (Previously Presented) The image pickup system 
according to claim 2, further comprising control means for 
controlling the precision verification means and the second 

-8- 



Application Serial No. 10/543,079 Customer No. 01933 

Response to Final Office Action 

interpolation means such that the operation of the precision 
verification means and the operation of the adjustment means can 
be stopped. 

14. (Previously Presented) The image pickup system 
according to claim 3, further comprising control means for 
controlling the precision verification means and the second 
interpolation means such that the operation of the second 
interpolation means and the operation of the precision 
verification means can be stopped, and for controlling the 
selection means such that when the operation of the second 
interpolation means and the operation of the precision 
verification means are stopped, the selection means is caused to 
select only the color signals that are interpolated by the first 
interpolation means. 

15. (Previously Presented) The image pickup system 
according to claim 12, wherein the control means comprises: 

information acquisition means for acquiring at least one 
type of information selected from a set comprising image quality 
information relating to the image quality of the image signals, 
image pickup mode information set in the image pickup system, and 
interpolation processing switching information that can be 
manually set, and 
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judgment means for judging whether or not the operations are 

10 to be stopped based on at least one type of information selected 

from a set comprising the image quality information, image pickup 
mode information, and interpolation processing switching 
information . 



16. (Currently Amended) A computer -readable recording 
medium having stored thereon an image processing program method, 
for an image pickup system, for processing 7 — by means of a 
computer, an image signal at each pixel which comprises a 
5 plurality of color signals, wherein the image pickup system 

comprises an image pickup element which outputs an image signal 
for each pixel from which -arrd one or more of the color signals 
are dropped out according to the location of the pixel, the image 
processing program causing the computer to function as method 
10 comprising : 

first interpolation means for interpolating , by a first 
interpolating unit of the image pickup system, the color signals 
dropped-out from the image signals by a first interpolation 
method; 

15 precision verification means for verifying , by a 

verification unit of the image pickup system, the interpolation 
precision based on the image signals and the color signals 
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interpolated by the first interpolation means interpolating unit ; 
and 

second interpolation means for interpolating , by a second 
interpolating unit of the image pickup system, the color signals 
dropped-out from the image signals by a second interpolation 
method that is different from the first interpolation method in 
cases where it is judged that the interpolation precision is 
insufficient . 



17. (Currently Amended) A c ompu t e r - r e adab 1 e recording 
medium having stored thereon an image processing program method, 
for an image pickup system, for processing -, — by means of a 
computer, an image signal at each pixel which comprises a 
5 plurality of color signals, wherein the image pickup system 

comprises an image pickup element which outputs an image signal 
for each pixel from which -arnd one or more of the color signals 
are dropped out according to the location of the pixel, the image 
processing program causing the computer to function as method 
10 comprising : 

separation means for separating , by a separating unit of the 
image pickup system, the image signals into first image signals 
and second image signals based on predetermined characteristics 
relating to the image signals; 



-11- 



Application Serial No. 10/543,079 Customer No. 01933 

Response to Final Office Action 

15 first — interpolation means — fxrr interpolating , by a first 

interpolating unit of the image pickup system, the color signals 
dropped-out from the first image signals by a first interpolation 
method; 

second interpolation means for interpolating , by a second 

20 interpolating unit of the image pickup system, the color signals 
dropped-out from the second image signals by a second 
interpolation method that is different from the first 
interpolation means method ; 

precision verification means — frerr verifying , by a 

25 verification unit of the image pickup system, the interpolation 
precision based on the first image signals and the color signals 
interpolated by the first interpolation means interpolating unit 
for the regions of the first image signals, and fxrr verifying , by 
the verification unit, the interpolation precision based on the 

30 second image signals and the color signals interpolated by the 

second interpolation means interpolating unit for ttre regions of 
the second image signals; and 

adjustment means for causing , by an adjustment unit of the 
image pickup system, interpolation processing of the dropped-out 

35 color signals to be performed again from the image signals by the 
second interpolation means interpolating unit when insufficient 
interpolation was performed by the first interpolation means 
interpolating unit , and -tor causing , by the adjustment unit. 
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interpolation processing of the dropped-out color signals to be 
40 performed again from the — image — signals by the first interpolation 
means interpolating unit when insufficient interpolation was 
performed by the second interpolation means interpolating unit , 
in cases where it is judged that the interpolation precision is 
insufficient . 



18. (Currently Amended) A computer -readable recording 
medium having stored thereon an image processing program method, 
for an image pickup system, for processing 7 — by means — erf — a 
computer, an image signal at each pixel which comprises a 
5 plurality of color signals, wherein the image pickup system 

comprises an image pickup element which outputs an image signal 
for each pixel from which ctrrd: one or more of the color signals 
are dropped out according to the location of the pixel, the image 
processing program causing the computer to function as method 
10 comprising : 

first interpolation means for interpolating , by a first 
interpolating unit of the image pickup system, the color signals 
dropped-out from the image signals by a first interpolation 
method; 

15 second interpolating means for interpolating , by a second 

interpolating unit of the image pickup system, the color signals 
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dropped-out from the image signals by a second interpolation 
method that is different from the first interpolation method; 
precision verification means for verifying , by a 

20 verification unit of the image pickup system, the interpolation 
precision based on the color signals, the color signals being 
interpolated by the first interpolation means interpolating unit 
and by the second interpolation means interpolating unit ; and 
selection means for selecting , by a selection unit of the 

25 image pickup system, color signals that have a higher 

interpolation precision between the color signals that are 
interpolated by the first interpolation means interpolating unit 
and the color signals that are interpolated by the second 
interpolation means interpolating unit . 



19. (Previously Presented) The image pickup system 
according to claim 2, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals; 

edge extraction means for extracting a plurality of edge 
intensities relating to predetermined directions from the pixels 
of interest within the extracted regions; 
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weighting calculation means for calculating weighting 
10 coefficients that are normalized from the extracted edge 
intensities ; 

interpolation signal calculation means for calculating a 
plurality of interpolation signals relating to predetermined 
directions from the pixels of interest within the extracted 
15 regions; and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest based on a plurality of 
weighting coefficients relating to the predetermined directions 
and a plurality of interpolation signals relating to the 
20 predetermined directions. 



20. (Previously Presented) The image pickup system 
according to claim 2, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals; and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest within the extracted regions by 
linear interpolation or cubic interpolation. 
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21. (Previously Presented) The image pickup system 
according to claim 2, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals; 

correlation calculation means for determining the 
correlation between the respective color signals within the 
extracting regions as a linear equation; and 

calculation means for calculating the dropped-out color 
10 signals based on the image signals and the correlation. 



22. (Previously Presented) The image pickup system 
according to claim 2, wherein the precision verification means 
comprises : 

correlation calculation means for determining correlation 
5 information relating to the correlations between the respective 
color signals for each predetermined region based on the image 
signals and the color signals interpolated by the first 
interpolation means, and 

correlation verification means for verifying the 
10 interpolation precision based on the correlation information. 
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23. (Previously Presented) The image pickup system 
according to claim 2, wherein the precision verification means 
comprises : 

hue calculation means for determining hue information for 
each pixel based on the image signals and the color signals 
interpolated by the first interpolation means, and 

hue verification means for verifying the interpolation 
precision based on the hue information. 

24. (Previously Presented) The image pickup system 
according to claim 2, wherein the precision verification means 
comprises : 

edge calculation means for determining edge information for 
each predetermined region based on the image signals and the 
color signals interpolated by the first interpolation means, and 

edge verification means for verifying the interpolation 
precision based on the edge information. 

25. (Previously Presented) The image pickup system 
according to claim 3, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
size centered on pixels of interest from the image signals; 
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edge extraction means for extracting a plurality of edge 
intensities relating to predetermined directions from the pixels 
of interest within the extracted regions; 

weighting calculation means for calculating weighting 
10 coefficients that are normalized from the extracted edge 
intensities ; 

interpolation signal calculation means for calculating a 
plurality of interpolation signals relating to predetermined 
directions from the pixels of interest within the extracted 
15 regions; and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest based on a plurality of 
weighting coefficients relating to the predetermined directions 
and a plurality of interpolation signals relating to the 
20 predetermined directions. 



26. (Previously Presented) The image pickup system 
according to claim 3, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
5 size centered on pixels of interest from the image signals; and 

calculation means for calculating the dropped-out color 
signals in the pixels of interest within the extracted regions by 
linear interpolation or cubic interpolation. 
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27. (Previously Presented) The image pickup system 
according to claim 3, wherein the first interpolation means or 
second interpolation means comprises: 

extraction means for extracting regions of a predetermined 
size centered on pixels of interest from the image signals; 

correlation calculation means for determining the 
correlation between the respective color signals within the 
extracting regions as a linear equation; and 

calculation means for calculating the dropped-out color 
signals based on the image signals and the correlation. 

28. (Previously Presented) The image pickup system 
according to claim 3, wherein the precision verification means 
comprises : 

correlation calculation means for determining correlation 
information relating to the correlations between the respective 
color signals for each predetermined region based on the image 
signals and the color signals interpolated by the first 
interpolation means, and 

correlation verification means for verifying the 
interpolation precision based on the correlation information. 
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29. (Previously Presented) The image pickup system 
according to claim 3, wherein the precision verification means 
comprises : 

hue calculation means for determining hue information for 
each pixel based on the image signals and the color signals 
interpolated by the first interpolation means, and 

hue verification means for verifying the interpolation 
precision based on the hue information. 

30. (Previously Presented) The image pickup system 
according to claim 3, wherein the precision verification means 
comprises : 

edge calculation means for determining edge information for 
each predetermined region based on the image signals and the 
color signals interpolated by the first interpolation means, and 

edge verification means for verifying the interpolation 
precision based on the edge information. 

31. (Previously Presented) The image pickup system 
according to claim 13, wherein the control means comprises: 

information acquisition means for acquiring at least one 
type of information selected from a set comprising image quality 
information relating to the image quality of the image signals, 
image pickup mode information set in the image pickup system, and 
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interpolation processing switching information that can be 
manually set, and 

10 judgment means for judging whether or not the operations are 

to be stopped based on at least one type of information selected 
from a set comprising the image quality information, image pickup 
mode information, and interpolation processing switching 
information . 



32. (Previously Presented) The image pickup system 
according to claim 14, wherein the control means comprises: 

information acquisition means for acquiring at least one 
type of information selected from a set comprising image quality 
5 information relating to the image quality of the image signals, 

image pickup mode information set in the image pickup system, and 
interpolation processing switching information that can be 
manually set, and 

judgment means for judging whether or not the operations are 
10 to be stopped based on at least one type of information selected 
from a set comprising the image quality information, image pickup 
mode information, and interpolation processing switching 
information . 
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